Skip to content

Swarming: Adds CountTasks endpoint & Refactors swarming Prpc request#5277

Open
IvanBM18 wants to merge 5 commits into
masterfrom
feature/swarming/count_task_requests
Open

Swarming: Adds CountTasks endpoint & Refactors swarming Prpc request#5277
IvanBM18 wants to merge 5 commits into
masterfrom
feature/swarming/count_task_requests

Conversation

@IvanBM18
Copy link
Copy Markdown
Collaborator

@IvanBM18 IvanBM18 commented May 14, 2026

As part of the Swarming backpressure initiative, we needed to launch request to the prpc's swarming.v2.Task/CountTasks endpoint so we can calculate the amount of fuzz tasks needed to schedule, to achieve this i moved the prpc request logic to its own module so that we can reuse code and keep responsibilities in separate files.

Changes

  • Created a new api.py file in the swarming module.
    • Moved the request logic from __init__.py to api.py
    • Removed push_swarming_task from init.py
    • Updated service.py to use SwarmingAPI directly.
    • Created a new class SwarmingAPI to handle the prpc request logic such as token/auth logic, swarming config handling, and any other request logic
  • Created new unit tests for this api and Updated existing unit tests.

Tests

  • Created unit tests for the new api & endpoint
  • Scheduled swarming jobs to verify that the refactor didn't broke the existing swarming logic

Note:

This PR is dependendant to this other PR any changes you see related to the proto files are to be reviewed in said pr.

@IvanBM18 IvanBM18 requested a review from a team as a code owner May 14, 2026 23:05
@IvanBM18 IvanBM18 changed the title feature/swarming/count task requests Swarming: Adds CountTasks endpoint & Refactors swarming Prpc request May 14, 2026
Copy link
Copy Markdown
Collaborator

@Xeicker Xeicker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future PRs, when you move code and then add functionality it is a good practice to do each of those steps in separate commits for an easier review

Comment thread src/clusterfuzz/_internal/swarming/api.py
Comment thread src/clusterfuzz/_internal/swarming/api.py Outdated
Comment thread src/clusterfuzz/_internal/swarming/api.py Outdated
Comment thread src/clusterfuzz/_internal/swarming/api.py
])
self.assertEqual(spec, expected_spec)

def test_push_swarming_task(self):
Copy link
Copy Markdown
Collaborator Author

@IvanBM18 IvanBM18 May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this test method is no longer needed here, in its place we added unit tests at the api_test that checks for the same functionality

@IvanBM18 IvanBM18 self-assigned this May 15, 2026
@IvanBM18 IvanBM18 added the swarming Changes related to the clusterfuzz-swarming integration label May 15, 2026
@IvanBM18 IvanBM18 force-pushed the feature/swarming/count_task_requests branch from d45af03 to 562ec1b Compare May 15, 2026 20:20
@IvanBM18 IvanBM18 force-pushed the feature/swarming/count_task_requests branch from 562ec1b to 57886fc Compare May 15, 2026 21:17
@IvanBM18 IvanBM18 requested a review from Xeicker May 15, 2026 21:41
Copy link
Copy Markdown
Collaborator

@Xeicker Xeicker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than some nits changes LGTM

message_body = json_format.MessageToJson(count_request)
logs.info(
"[Swarming] Counting tasks in queue",
url=self._base_url,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

Might be useful to have the complete url in the log

logs.info("[Swarming] Counting tasks", url=url, body=message_body)

response = utils.post_url(url=url, data=message_body, headers=headers)
response = self._make_request('swarming.v2.Tasks/CountTasks', message_body)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

It sounds to me that endpoints could be module constants

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

swarming Changes related to the clusterfuzz-swarming integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants